Thoroughly testing a computerized application generally requires testing every potential flow through the application. A traditional approach that meets this requirement is the creation of a suite of automated test scripts that are generated with prior knowledge of the application's desired behavior. Whenever modifications are made to the application, the suite of test scripts can be run against the application to assure that the modifications have not created a bug or fatal flaw in the system. Therefore, when modifications are made that intentionally change the desired behavior of the application, the suite of test scripts can be updated so that they properly test the application.
If many different applications need to be examined by the same testing facility, the generation of the multiple suites of test scripts needed to test all the different applications can be prohibitively time consuming. In addition, maintaining the test scripts in step with intentional changes in the requirements and desired behavior of the applications presents a difficult challenge in terms of coordinating effort between the teams that are responsible for the test scripts and those that are responsible for updating the applications. This is particularly true in situations in which different companies are responsible for testing and designing the applications. For example, one company may sell devices that facilitate the operation of a platform of software applications, while a broad base of third parties design software applications for that platform. If the company that administrates the platform takes responsibility for the stability of all applications that are available for operation on the platform, testing the applications can be difficult because the company will not have a body of prior knowledge regarding how the applications should function before the applications are received for testing.